Emotion Extraction from paintings based on Luscher Color test and Culture Technology (CT)

 

 

Babak Ranjgar, Mahdi Khoshlahjeh Azar, Abolghasem Sadeghi-Niaraki

 

Department of Geodesy and Geomatic, K. N. Toosi University of Technology

1346, ValiAsr Street, Mirdamad cross

Tehran, Iran

[email protected], [email protected], [email protected]

 

 

 

Abstract

 

Paintings are part of a nation’s culture, as different cultures have specific styles of painting. When human beings look at paintings they feel certain emotions. In this paper, we have presented a novel approach for emotion recognition in paintings based on Luscher Color Test and Culture Technology (CT) concept. To the best of our knowledge this psychological theory have never been used in this domain. This paper is a first attempt to use this untouched color-mood theory in psychology.

 

Keywords- Luscher Color Test, Painting, Color, Emotion, Culture Technology

 

1. Introduction

                                                                      

When people look at paintings, they feel certain emotions. In fact, in some painting styles, like Expressionism, the artist tries to depict a special emotion. Why is it important to study about paintings? Different cultures have specific styles of painting. Paintings are part of a country or nation’s artistic and cultural heritage, depicting the identity and history of those people. By the advent of new technologies, it is possible to better preserve and understand cultural content. As a matter of fact, this idea was first developed by Professor Kwangyun Wohn as Culture Technology (CT) in 1994. Its goal is to take advantage of technology to better study, analyze, and disseminate culture.

In this paper we are going to study on the extraction of emotions evoked by paintings in the field of CT [8] [9]. Due to psychological studies [1] [2] [3] [4] there is a great relationship between colors and evoked emotion in human and it has a universal trend [5]. Therefore we are going to use colors that shape the paintings to extract emotions.

We are going to use Luscher Color Test [6] as psychological basis for extraction emotions evoked by paintings. Luscher Color Test is a simple yet powerful test to understand personality of human. In this test, eight colors is used that consists of four basic color (blue, green, red, and yellow) and four auxiliary colors (purple, brown, gray, and black) (fig1.) and subject’s color preference in choosing the colors is important in revealing their characteristics. Our idea is to capture these eight colors in paintings and arrange them based on the area they cover in the paintings. After obtaining color preference of the painting then we can extract the emotion of a painting.

 

Figure 1. 8 colors in Luscher Color Test

The remainder of this paper is as follows. In section 2 we are going to discuss Luscher Color Test in more detail. We explain our approach of extracting paintings’ color palette as the input for the Luscher Color Test. Then we present results of our proposed method and its evaluation in section 3. At last but not least we provide our ideas, summary of the work and our future goal as conclusion in section 4.

 

2. The Luscher Color Test

 

The Luscher Color Test is devised by Dr. Max Luscher in Basel, Switzerland. Although this is designed in a simple way, it is a powerful tool for personality assessment. In the ordinary form of the test, a subject puts the eight color cards in order of their preference. The basic colors represent good personality features, while the auxiliary colors demonstrate less good personality features. In this test each color has special meanings but we have extracted eight main emotion for eight colors that are presented in table1.

Table 1. Emotions related to each color in Luscher Color Test

 

Also Dr. Luscher created four layers and each layer is for two colors. First two colors are related to first layer and they are the colors that the subject likes the most. Third and fourth colors are related to the second layer in Luscher test which reflects the current state of the subject. Fifth and sixth colors are related to the third layer. This layer shows the characteristics that subject has put aside temporarily and may be used at the right time. The fourth layer is related to the last two colors which are hated the most by the user. These layers are like 2D matrices, containing 56 elements (all possible two color combinations), so the order of the two colors in each layer is also important. Each element contains statements about the subject selecting those two colors, in order to use these layers we have created four 3-dimentional matrices, the row and column dimensions consists of eight colors and the third dimension has eight emotions stated above. Elements of these matrices were rated by experienced psychologists to form our knowledge base.

 

3. Emotion extraction and experiment results

 

Now we have created a knowledge base with four layers and an input color palette of the eight colors in the Luscher test is needed for a painting to extract the vectors of emotions from each layer.

For dealing with colors, a color space should be chosen, RGB color space is frequently used in image processing, but this color space is not similar to how human perceives colors, hence we use HSV color model that is suitable for this work and also it has the capability of thresholding. We extract the eight colors in paintings and sort them from the mostly used to the least used color, which is a naïve idea that the most used color affects human perception the most. Then each two color of the color palette created from the painting, goes to the related layer and the corresponding emotion vector is extracted. So we have four of these emotion vectors extracted from each layer, in this study we naively calculated the mean of these four vectors in order to obtain a single vector containing the eight emotions.

For experiment purpose and evaluation of our method we need a ground truth dataset of paintings. Except IAPS, which is a dataset of images, there is no special painting dataset. Furthermore results of any other work are different with the results of our method because the Luscher Color Test is not used in this domain. We chose the 16 paintings in Kobayashi’s work. Kobayashi devised a color image scale [8] and showed the paintings on that scale. This color image scale contains many emotional words and each have a specific coordinate. Therefore we have to map our emotion vector to the Kobayashi’s cool-warm and soft-hard emotion coordinates. For this purpose we created a set of rules that are conditions for the emotional words. There are two examples below.

	If	(calmness is high & wish is high &   despair is low)
	Then	Refreshing
	If	(calmness is low & passion is low & physical pain is low)
	Then	mild

 If the conditions are met, then we put a point on the color image scale for each word, at the end of this procedure we have a map of points related to the paintings.

 

Figure 2. Overall procedure of the proposed method

 

At the final stage of our work we use a 2d kernel density estimation [7] in order to specify the position of the paintings. In fig. the whole procedure of our method is demonstrated.

We put each 16 paintings in this procedure and calculated their position on the color image scale by our method. Kang et al. used machine learning and based on the three color combinations in Kobayashi’s work have specified the position of the 16 painting [8]. Comparison of the three works is depicted in fig.3, fig.4, and fig.5.

Figure 3. Kobayashi’s ground truth

Figure 4. Kang et al. [8]

Figure 5. Our method

 

4. Conclusion

 

In this paper, we used Luscher Color Test to extract emotions from paintings. This test has eight colors, four basic and four auxiliary colors. Color preference is the key parameter in Luscher test and also two color combinations. Each two color in the test is related to a layer. We asked experienced psychologists to read the statements in the four layers and rate the eight emotions in Luscher test. We created four 3D matrices as our knowledge base and it requires an ordered color palette as an input. Each two color from the ordered palette goes to the related layer respectively and four emotion vectors are extracted as a result. In this work we naively calculated the mean of these four vectors to have a single emotion vector. For experiment purpose we chose the 16 paintings in Kobayashi’s work. As the color image scale and emotions in Luscher color Test are not directly comparable, so we created a set of rules to map the emotional vector to the color image scale.

Our method based on Luscher Color Test showed promising results, yet it is less good than original work by Kobayashi. This might be due to the errors that are inevitable when we map the emotional vector to the color image scale. Also as stated in [8] the colors of the digital image acquired now are different to those used by Kobayashi. This work was the first step towards studying our countries’ rich historic heritage of paintings. For future work we are going to use more ground truth paintings and try to find better ways of comparison so no additional errors can affect the results.

 

Acknowledgment

 

We sincerely thank all the people who contributed to this work. This research was done as a part of international Culture Technology dual degree program between K. N. Toosi University of Technology and KAIST.

 

 

References

 

[1]   Wright, B., & Rainwater, L. (1962). The meanings of color. The Journal of general psychology, 67(1), 89-99.

[2]   Naz, K., & Epps, H. (2004). Relationship between color and emotion: A study of college students. College Student J, 38(3), 396.

[3]   Valdez, P., & Mehrabian, A. (1994). Effects of color on emotions. Journal of experimental psychology: General, 123(4), 394.

[4]   Kobayashi, S. (1981). The aim and method of the color image scale. Color research & application, 6(2), 93-107.

[5]   Adams, F. M., & Osgood, C. E. (1973). A cross-cultural study of the affective meanings of color. Journal of cross-cultural psychology, 4(2), 135-156.

[6]   Lüscher, M. (1990). The Luscher color test. Simon and Schuster.

[7]   Shimazaki, H., & Shinomoto, S. (2010). Kernel bandwidth optimization in spike rate estimation. Journal of computational neuroscience, 29(1-2), 171-182.

[8]  Kang, D., Shim, H., & Yoon, K. (2015, January). Mood from painting: Estimating the mood of painting by using color image scale. In Frontiers of Computer Vision (FCV), 2015 21st Korea-Japan Joint Workshop on (pp. 1-4). IEEE.

[9]   Lee, T., Lim, H., Kim, D. W., Hwang, S., & Yoon, K. (2016, November). System for matching paintings with music based on emotions. In SIGGRAPH ASIA 2016 Technical Briefs (p. 31). ACM